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ABSTRACT 


The Navy Supply Systems Command holds $21 billion of inventory across 
430,000 repair part line items in support of ongoing naval operations. Selecting the 
correct reorder point for each of those line items requires a careful balance between tying 
up precious funding by holding too much inventory that may not be issued, and not 
holding enough inventory to meet customer demands in a timely fashion. This thesis 
compares three different methods for selecting the reorder point. The first method is a 
calculation that offers no attempt at optimization, but is simple to understand. The second 
method is provided by a contractor, and uses some optimization with unknown 
algorithmic details. The last method is a mixed-integer, linear optimization model. 
Comparative Inventory Simulation, a discrete event simulation model, is designed to find 
fill rates achieved for each National Item Identification Number under the different 
reorder methods. We find that average fill rates are higher in 22 out of 24 cases, and 
average backorder lengths are up to 50% shorter, using the last method. 
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THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may 
not have been exercised for all cases of interest. While every effort has been made within 
the time available to ensure that the programs are free of computational and logic errors, 
they cannot be considered validated. Any application of these programs without 
additional verification is at the risk of the user. 
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EXECUTIVE SUMMARY 


The Navy Supply Systems Command (NAVSUP) holds $21 billion of inventory 
across 430,000 repair part line items in support of ongoing naval operations. In an 
unconstrained world, NAVSUP would hold enough inventory to support repair of all 
naval systems through their entire life cycle; unfortunately, budget and storage 
constraints force NAVSUP to carry a limited number of repair parts and consumable 
items known as NIINs (National Item Identification Numbers) in its inventory. This 
process involves the selection, for each NUN, of an inventory position quantity at which 
to reorder the NIIN, known as the “reorder point.” Determination of optimal reorder 
points for all NIINs considered is a critical factor in supporting the warfighter and 
maintaining material readiness. 

This thesis compares three methods for selecting reorder points. A first method 
suggested by NAVSUP is the use of a simple calculation (SC) based on mean and 
standard deviation of lead-time demand. The second method is a contractor-provided tool 
called Service Planning and Optimization (SPO), currently in use at NAVSUP, which 
algorithmic details are unknown. The last method is the Wholesale Inventory 
Optimization Model (WIOM), a mixed-integer, linear optimization program recently 
developed at the Naval Postgraduate School. This thesis explores the effectiveness of the 
three different methods for calculating reorder points by developing a discrete-event 
simulation tool called Comparative Inventory Simulation (CIS). 

CIS is designed to model wholesale demands, issues, and reorders. CIS can be 
broken down into ten different events (such as order placing and receive, demand, etc.). 
As events occur, they modify the value of variables in the system (such as inventory 
levels, backorders, etc.) and schedule future events. Time proceeds in variable increments 
to the time of the next event. CIS uses a number of inputs from each NUN, the most 
important two being the reorder point (as detennined by one of the three methods) and 
the demand probability distribution (PD) function describing the number of demand 
events that occur during a time period. This PD is generated either from a parametric fit 

(based on mean and variance of the demand), or from an empirical distribution (based on 
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historical monthly demand). Using the PD, CIS schedules events such as filling demands, 
and placing and receiving reorders. In this process, CIS counts the number of backorders 
for each NUN, which allows the calculation of fill rates. At the conclusion of the 
simulation, CIS reports the fill rate and the mean length of time backordered for each 
NIIN. 

Five different datasets supplied by NAVSUP are explored via simulation: S5S8, 
aviation consumables, aviation repairables, maritime consumables, and maritime 
repairables. Each dataset contains a variety of high- and low-demand items as well as 
different PDs representing the demand patterns. The number of NIINs in each dataset 
ranges from 249 to 9,601. 

CIS can use an initial “warmup” period in order to determine the time required to 
overcome initialization biases (due to initial inventories). We conduct analysis of 
replications with and without using the wannup period, and find that in both cases fill 
rates achieve steady state (remain nearly constant within a 1% band) around the 400,000 
day in the simulation. 

The primary method of evaluating the effectiveness of the three methods of 
reorder point selection has been by comparing the fill rates achieved by simulating the 
demands for each NIIN. For each dataset, the overall, average fill rate and weighted fill 
rate are calculated using both empirical and parametric-fit PDs resulting in 24 total cases. 
Simulated fill-rates using WIOM’s reorder points lead to the highest observed fill rates in 
22 out of the 24 cases. The fill rates resulting from the use of the SC reorder points were 
only higher than SPO in a single case. 

A second consideration is that WIOM’s ability to estimate the fill rates is mixed 
and largely dependent upon the PD used for each NIIN. WIOM estimates the fill-rates 
within 2% of the simulated value for approximately 25% of the NIINs, and CIS indicates 
that WIOM is twice as likely to over-estimate as to under-estimate the fill-rate. When the 
demand PDs are fit to a Poisson distribution, WIOM is generally accurate. Fits made with 
the gamma PD are consistently less accurate. 



CIS has uncovered significant differences in the mean length of backorders 
between reorder points selected by WIOM and SPO. Simulated lengths are up to 50% 
shorter when using WIOM. It appears SPO places the reorder points for some NIINs too 
high at the cost of not being able to buy sufficient safety stock for other NIINs. WIOM 
spreads the safety stock more evenly throughout the NIINs. As a result, WIOM has fewer 
lengthy backorders. 
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I. INTRODUCTION 


A. BACKGROUND 

The Navy Supply Systems Command (NAVSUP) holds $21 billion of inventory 
across 430,000 repair part line items in support of ongoing naval operations. The Chief of 
Naval Operations has directed all NAVSUP activities to establish sparing requirements in 
order to meet readiness objectives (Office of the Chief of Naval Operations 2011). 
Finding the correct amount of spares to hold for each line item under uncertain demand 
and financial constraints is a critical factor in determining how effectively NAVSUP can 
support the warfighter (Silver 1998). In an unconstrained world, NAVSUP would hold 
enough inventory to support repair of all naval systems through their entire life cycle; 
unfortunately, budget and storage constraints force NAVSUP to carry a diminished 
number of repair parts in its inventory and select an inventory quantity at which to 
reorder the item. Determination of the best reorder point for each repair part is a critical 
factor in supporting the warfighter and maintaining material readiness. 

NAVSUP considers fill rate to be the primary metric to measure the effectiveness 
of the inventory policy at the wholesale level. Fill rate is defined using the following 
equation: 

total demands - number backordered 

till rate =- 

total demands 

This formula can be applied to a specific National Item Identification Number 
(NUN) or to a group of NIINs. Each NIIN is assigned a fill-rate target based on the 
relative criticality of the item. For instance, NAVSUP may assign a circuit card used to 
repair a radar a 90% fill-rate target while a plastic gauge cover may be assigned a 70% 
fill-rate target. By assigning different fill-rate targets to different NIINs, NAVSUP is able 
to prioritize those items that must be available for issue above those items for which 
some amount of delay is acceptable. 

The most straightforward method for calculating the reorder point uses the simple 
calculation (SC) as follows (Oswald 2014): 
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reorder point = mean demand over lead-time + Wvariance of the lead-time demand 

In SC, “t” is a value selected to make the reorder points small enough such that the total 
sum of the value of the safety stock inventory met the budget constraint. The value of “f ’ 
is adjusted upward or downward as appropriate until the budget constraint is achieved. As 
neither fill rate nor any other metric are considered in the selection of “t,” the reorder 
points, while easy to calculate, are clearly not optimal. 

NAVSUP currently uses Morris Cohen Associates Service Planning and 
Optimization (SPO) model to determine the number of each line item to carry in its 
inventory. As a contractor-provided solution, SPO is largely a “black box,” and the 
details of how the optimum inventory levels are determined are unknown. Additionally, 
anecdotal observations suggest that SPO favors buying inexpensive items to maximize 
fill rate while minimizing costs. By exceeding fill-rate goals with inexpensive items, SPO 
is able to maintain a high overall average fill rate while falling short on the fill-rate goals 
of expensive items. While this may make perfect sense in a business setting where a 
corporation seeks to maximize profit, it is preferable to meet all fill rate goals (as allowed 
by the constraints) in a military readiness environment. As a result, individual item 
managers must manually adjust reorder point for some NIINs further degrading the 
optimality of the solution. 

In an effort to provide a lower-cost alternative to SPO, Salmeron and Craparo 
(2014) developed the Wholesale Inventory Optimization Model (WIOM). WIOM 
simultaneously selects reorder values across all NIINs with the goal of minimizing 
weighted deviations from fill-rate targets. It uses historic data about each NUN as input 
into a General Algebraic Modeling Language implementation of a mixed-integer 
programming (MIP) model and produces a reorder point for each NIIN as output. 
Additional information about WIOM is located in Chapter II. 

B. OBJECTIVES 

This research develops Comparative Inventory Simulation (CIS), a discrete-event 
simulation of demands and replenishments in order to answer the following questions: 
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• Are the fill-rate estimation calculations performed by WIOM accurately 
reflected by the simulated fill-rates? 

• How do the reorder points selected by SC, SPO and WIOM affect the 
overall fill-rate? 

• Does the selection of the reorder point by SPO or WIOM change the 
amount of time it takes for a backordered NIIN to be filled? 

• Does the selection of a demand probability distribution affect fill-rate 
deviation? 

This thesis provides qualitative and quantitative information to help NAVSUP 
explore the effects of using different methods to detennine the reorder point. The result 
of this thesis will assist NAVSUP in determining the strengths and weaknesses of the SC, 
SPO, and the more recently developed WIOM, and inform future decision making with 
regard to different reorder point strategies. 

C. THESIS FLOW 

Chapter II contains the literature review and explores work that has already been 
completed on similar problems. Chapter III describes the assumptions used in developing 
CIS, CIS itself and the datasets used to explore the problems. Chapter IV contains the 
results of the experiments. Chapter V draws conclusions from the results and provides 
some recommendations for follow-on work. 
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II. LITERATURE REVIEW 


It is helpful to conduct a review of previous studies and publications to establish a 
foundation upon which the simulation can be constructed and that data can be analyzed. 
First, we examine different inventory policies that could be used to support the Navy’s 
goals. We will look at how discrete-event simulation could be applied to the problem. We 
conclude the literature review with a survey of the work concerning the analysis of 
simulation models. The definition of inventory terms used in this section are located in 
Appendix A. 

A. INVENTORY POLICY 

One of the fundamental problems faced by inventory managers is when a 
replenishment order should be placed for an item in their inventory (Silver et al. 1998). 
Placing an order too soon results in excess inventory and ties up capital that could be 
used to hold other items. Placing an order too late causes inventory shortfalls and 
subsequent delays in providing the end-user with the material they require. To address 
this problem, NAVSUP utilizes an order-point, order-quantity (s, Q) system. 

The order-point, order-quantity inventory policy is a continuous review system 
where the inventory position for a particular NUN has the ability to trigger a reorder 
event. In this system, inventory position is defined as the quantity of a particular item 
held in inventory plus the quantity currently on order minus the backordered quantity. As 
items are issued from inventory, inventory position decreases. When inventory position is 
equal to the reorder point (s), a reorder event for quantity Q occurs (see Figure 1). 

Silver et al. (1998) notes several advantages achieved when the order-point, 
order-quantity system is used. Since inventory position considers ordered but unreceived 
items, this inventory policy precludes placing an extra order today when material arriving 
tomorrow would be sufficient to cover the shortage. Additionally, the system can be 
easily visualized. The primary drawback to the order-point, order-quantity system is that 
there is not a defined procedure to handle those cases when individual demand quantities 
exceed the reorder quantity such that the reorder of Q items will not raise inventory 
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position above the reorder point. For this thesis, orders are assumed to always be for a 
single quantity so the drawback is not a problem. 


Figure 1. Example of an Order-point, Order-quantity (s, Q) System 
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Source: Silver EA, Pkye DF, Peterson R. (1998). Inventory’ Management and Production 
Planning and Scheduling. (John Wiley & Sons, New York). The solid line in this figure 
represents on-hand inventory, and the dotted line represents inventory position. Items are 
issued from the inventory until the inventory position drops below the reorder point (s) at 
time A when an order in the amount of the reorder quantity (Q) is placed. At time B, the 
outstanding order is received. Since there are not any outstanding orders, inventory 
position is equal to on-hand inventory. 


B. DISCRETE EVENT SIMULATION 

Discrete event simulation utilizes state changes that occur at a discrete set of 
points along a time axis (Schruben 1983, Law 2007, Sanchez 2007). As events occur, 
they modify the value of variables that describe the system. Events may also schedule 
future events to occur. Events are held in an event list as a data pair while waiting 
execution. The first part of the pair is the name of the scheduled event. The second part of 
the pair is the time at which the event will occur. This list is maintained in a time-sorted 
queue, and at the conclusion of the current event, the next event is popped off the queue 
and simulation time is advanced to the time of next scheduled event. Thus, the simulation 
proceeds from one event to another. Since state transitions only occur at events, the time 
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between events can be passed over without concern. Note that simulated time does not 
proceed in fixed but variable time increments that are detennined by when events are 
scheduled. 


The relationship between events and the changes that they make on the system 
can be described by event graphs. Event graphs consist of two different elements: event 
nodes and scheduling edges. Each event node corresponds with a state transition that 
transpires in the system. During an event, state variables are changed according to the 
state transition function for that event. Event nodes are connected by scheduling edges. 
These edges can be marked with conditional statements and time delays. If the scheduling 
edge has a conditional statement, the connected event node will only be scheduled if the 
conditional statement is true. Conditional statements are represented on the event graph 
as an equation or inequality in parentheses. If the scheduling edge has a time delay, the 
connected event node will be scheduled to occur after the time delay has elapsed. Time 
delays are written as the letter t with a subscript. See Figures 2 and 3. 


Figure 2. Simple Event Graph 

(i) 



Source: Schruben, Lee. 1983. Simulation Modeling with Event Graphs. In 
Communications of the ACM 26 (11): 957-963.This event graph indicates that t time 
units after the occurrence of event j, event k will be scheduled to occur providing 
condition (i) holds at the time event j occurs. 
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Figure 3. Example of an Event Graph 



In this event graph, arrival are scheduled every t a time units. When an arrival occurs, a 
start service event is scheduled if Q is less than one. If Q is one or greater the arrival is 
discarded. Upon the start service event, Q is increased by one and the end service event is 
scheduled t s time units later. When the end service event occurs, Q is decreased by one. 
This event graph represents a system where a person arrives for service. If they can 
immediately be helped, service begins. If they would have to wait, they leave. 


C. TYPES OF SIMULATION MODELS 

There are two different types of simulation models that could be considered for 
inventory analysis: terminating and nonterminating. A terminating simulation is one for 
which there is a natural time horizon over which the simulation should be run. The 
simulation ends when one of three criteria are met: the system is cleared out, the point at 
which no further useful infonnation is obtained, or a time specified by management 
mandate (Law 2007). A system is cleared out is when the last event occurs and the queue 
is empty. Suppose a store has set hours of operation. The simulation would start when the 
store opens, and the simulation would terminate when the store closes. An example of a 
simulation that terminates without further infonnation would be a simulation of a race. 
Data would be collected up to the point when the last racer crossed the finish line, at 
which point additional data collection would be meaningless. An example of a simulation 
that terminates on a time specified by mandate is a simulation, which is to collect data for 
seven days. After the seventh day, the simulation ends. 

A nonterminating simulation is one for which there is no event to specify the 
length of a run (Law 2007). This is most useful when there is concern about the long¬ 
term behavior of a system but there no specific definition of long-term exists. In these 
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simulations, the desire is to find the steady state of the variables of the measures of 
performance. Since this thesis is concerned with finding the fill-rate achieved by the 
selection of different reorder points, a nontenninating simulation is the natural selection 
for implementation. 

D. ANALYSIS OF SIMULATION MODELS 

There are several considerations to be taken into account when conducting 
analysis of a simulation model. The first problem is the sensitivity of the simulation to the 
starting conditions. At the beginning of the simulation, a steady state has not yet been 
achieved; therefore, the data collected has been influenced by whatever values have been 
initially assigned to the variables. Law (2007) described this phenomenon as the problem 
of the initial transient and proposed suspending data collection until a warmup period has 
passed. The problem then became detennining the length of the wannup period. Welch 
(1983) proposed a solution to determining the length of the wannup period through a 
four-step process. In step one, multiple replications of the simulation were generated. In 
step two, the i th observation from each replication were averaged to find the mean value 
for that observation. In step three, a weighted moving average was computed from the 
averaged values to smooth the high frequency oscillations. In step four, the weighted 
moving average was plotted and the point at which convergence has appeared is selected 
as the warmup period. 

The second consideration concerns the effect of the input parameters on the 
output of the simulation. If there were a single input parameter into the model, that 
parameter could be varied to gauge its effect. However, if multiple input variables need to 
be tested, other means need to be employed. A 2 k factorial design allows for two different 
levels to be selected for k inputs. The simulation is then run for each combination of 
factor levels for each input parameter. There are two problems with this approach. The 
first is the restriction to two levels per input parameter, many simulations would find this 
too limiting. The second is the number of simulation runs required to cover all 2 k cases 
with enough iterations to construct confidence intervals. The computation time is simply 
unacceptable. 
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An m k factorial design removes the two-level restriction posed by the 2 k level 
design (Sanchez 2008). This design allows for m levels to be selected for each input 
parameter thereby greatly increasing the solution space that can be searched. However, 
this increases the number of cases that have to be considered and is even more 
computationally expensive than the 2 k factor design. 

Nearly Orthogonal Latin Hypercubes offer an alternative approach giving similar 
solution space exploring qualities as the m k factorial design with a significantly 
diminished number of samples. Cioppa and Lucas (2007) proposed a method for 
constructing Nearly Orthogonal Latin Hypercubes. 

E. WHOLESALE INVENTORY OPTIMIZATION MODEL 

WIOM is the result of an effort to provide an alternative method to NAVSUP in 
order to find optimal reorder points for their wholesale inventory. This section will 
explore how WIOM uses optimization to determine these values. 

1. Optimization 

There are many instances where a person seeks to maximize (or minimize) a 
particular result. “Optimization models represent problem choices as decision variables 
and seek values that maximize or minimize objective functions of the decision variables 
subject to constraints on variable values expressing the limits on possible decision 
choices” (Rardin 1998). The objective function measures the effectiveness of the values 
of the decision variables. The value of the decision variables are limited by a set of 
equalities and inequalities called the constraints. The set of values for the decision 
variables which maximize (or minimize) the value of the objective function while 
satisfying the constraints represents the optimal solution. 

There are three categories of optimization defined by the characteristics of the 
objective function and the constraints. A linear program is one in which all of the 
decision variables are continuous and both the constraints and the objective function are 
linear functions of the decision variables. A MIP has the characteristics of a linear 
program but relaxes the requirement for continuity by allowing the decision variables to 
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be restricted to integer values. A nonlinear program allows for nonlinear functions to 
appear in the objective function as well as the constraints. 


2. The WIOM Model 

According to Salmeron and Craparo (2014), “WIOM seeks reorder points that are 
globally optimal for all NIINs simultaneously considered, while observing a number of 
constraints.” This optimization model was implemented as a MIP. The MIP seeks to 
minimize the sum of the penalties assessed when the NIIN’s target fill rate is missed. 
This necessitates finding an estimate for fill rate, which, in turn, requires an estimate for 
the demand probability distribution function (DPDF). 

One way to generate the DPDF is to leverage the empirical distribution as 
represented by 60 months of demand history for each NUN. This distribution is unique 
for each NUN and each value represents the number of demands for a single quantity in a 
particular month. A three step process coverts the empirical distribution to DPDF. In step 
one, the monthly demand is uniformly distributed during the month. In step two, the 
months are converted to lead-times. Additionally, the lead-time demand x and the cycle 
length c are calculated. In step three, the lead-time demand is converted to cycle 
demand and the occurrences of each demand level are counted to find the DPDF. An 
example of this process is in Figure 4 and the resulting distribution is in Figure 5. 
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Figure 4. The Three Steps to Calculate the DPDF 

Monthly Demand: | l| 2| 3| o| l| o| l| l| 

Lead-time: 2 months 
Q: 1 



Step 1 
Month 
Step 2 
Lead-times 
Step 3 


123456789 10 Cycles 

This example uses monthly demand data with a two-month lead-time. An ‘x’ in the 
image represents where a demand occurs in time. In the first step, the monthly demand is 
uniformly distributed during the corresponding month. In step two, the months are 
converted into lead-times. Since the length of a lead-time is two months, the eight months 
are converted into four lead-times. We then calculate the lead-time demand X and the 

zv 10 demands demands i 2.5 cycles 

cycle length C. X = = 2.5 c — max { 1, } — max { 1, } — 2.5 - The 

4 lead-times lead-time Q 1 lead-time 

occurrences at each demand level are counted to find the DPDF in Figure 5. 


Figure 5. Resulting DPDF 
Demand Probability Distribution Function 

0.45 
0.4 
0.35 
>. 0.3 
I 0.25 

03 

| 0.2 
0.15 
0.1 
0.05 
0 


This graph captures the cyclic demand from Figure 4. Out of the 10 cycles, three 
experienced zero demands, four experienced four demands and three experienced two 
demands. 
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Ideally, WIOM would always be able to use an empirical distribution representing 
the demand during the cycle time. However, three reasons prevent the empirical 
distribution to be used for some NIINs. First, monthly demand data are not available for 
all NIINs. Secondly, some NIINs have so little demand that the empirical distribution 
cannot be applied. Lastly, some NIINs had fractional values for demands during the 
month. For these cases, we use the expected demand x and standard deviation en¬ 
during the lead-time. WIOM coverts the lead-time parameters into cycle-time parameters 
during a preprocessing step. WIOM then converts these cycle-time parameters into a 
DPDF by applying a parametric fit to a prefixed standard distribution suggested by 
NAVSUP. The details of these fits can be found in the WIOM documentation, with a 
summary provided by Table 1. 


Table 1. Probability Distributions and Estimators to Approximate Lead- 

time Demand 


Prob. 

Dist. 

Name (Parameters) 

Mean 

Variance 

Variance/Mean 

Parameter Estimation 
(Based on x,cr) 

1 

Poisson (2) 

2 

2 

1 

A = x or A = d 2 

2 

Negative Binomial (r.p) 
(r=# failures to stop) 
(Generalized Neg. Bin.) 

ip / (1 -p) 

tp/(l-p) J 

l/(l-p)>l 

P = 1-^t; >' = -72—r 
cr* <7* - x 

3 

Normal (//. a) 

P 

cr 

(T / p 

II 

b 

■H 

II 

4 

Binomial (n.p) 

tip 

np(l-p) 

\-p<\ 

« 2 -O 

. , cr „ x* 

P~l .: n =. 

X X - cr* 

5 

Gamma (k. 6) 

(k=shape, #=scale) 

kd 

W 

e 

(7* X 

6 

Deterministic (p ) 

P 

0 

0 

p-x 


Source: Salmeron J, Craparo E (2014) Wholesale Inventory Optimization Models. 
Working paper, Naval Postgraduate School. 


The estimated fill-rate calculation used in WIOM was inspired by Silver et al. 
(1998). However, the data provided by NAVSUP violated two of the assumptions of 
Silver’s calculation (Salmeron et al. 2014). The first was that the average level of 
backorders could be significant when compared with the average level of on-hand stock. 
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The second was that multiple cycles could occur during a single lead-time. These two 
issues can be corrected by modifying Silver’s calculation by substituting cycle-time 
demand for lead-time demand. The resulting equation allows for WIOM to estimate fill- 
rate during optimization. 
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III. COMPARATIVE INVENTORY SIMULATION 
DEVELOPMENT AND DATA SETS 


The CIS is intended to detennine the fill-rate achieved for a NIIN given a 
particular cycle-time demand probability distribution function, reorder quantity and 
reorder point. The Python computer language is used to implement the discrete event 
simulation. 

A. ASSUMPTIONS 

In order to develop CIS and test the datasets, several assumptions are required: 

• The probability distribution functions determined by WIOM accurately 
reflect the actual demand distributions. 

• When continuous distributions are used to determine the quantity 
demanded in a time period, the resulting random variate can be rounded. 

• Lead-times for each NIIN are known and constant. 

• Demand for each NIIN is independent of demands for all other NIINs. 

• Individual demands are for one unit of the NIIN. 

• All demands have equal priority and will be filled in the order received. 

B. CIS FORMULATION 

1. Diagram and Description of Events 

The simulation can be represented in an event graph containing ten different 
events. The following notation (for a given NIIN i) is helpful in understanding the event 
graph and a definition of these tenns is in Appendix A: 

• t f : the length of a lead-time (days) 

• t\ : the length of a cycle (days) 

• y : the reorder point (units of issue) 

• Qi : the reorder quantity (units of issue) 

• L : the on-hand inventory quantity (units of issue) 
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• a j : the number of randomly determined demands in a particular cycle 
(units of issue) 

• BB j : the number of items currently backordered (units of issue) 

• OO i : the number of items currently on order (units of issue) 

The event graph is shown in Figure 6 and a description of each event follows: 

Run: The simulation begins with the run event scheduled. Upon execution, the run 
event sets the starting on-hand inventory level equal to the sum of the reorder point and 
the reorder quantity. This starting quantity represents the maximum amount of inventory 
that could be held at one time (a reorder has been received during a time when there were 
not any demands). While any amount greater than the reorder point could have been 
selected, this quantity was convenient and will later be shown not to have an impact on 
the results. 

Cycle: At the completion of the run event, the cycle event is scheduled. In a 
traditional inventory model, the cycle is defined by the length of the lead-time (Silver et 
al. 1998). However, this definition is founded upon the assumption that the reorder 
quantity is greater than the lead-time demand. This assumption does not hold for the 
NAVSUP data. Therefore, in this simulation the length of a cycle for NIIN i is defined by 
■ t 

t, = - j -- where t t is the lead-time, x, is lead-time demand and Q t is the reorder 

h A 
max 1, — 

l QJ 

size. The cycle event first determines the number of demand events that will transpire 
during the cycle according to the probability distribution function related to the NIIN. It 
then schedules both the “schedule cycle” and the “schedule demand” event. 

Schedule Cycle: The schedule cycle event occurs t’ time units after the 
corresponding cycle event. This event checks to see if there is still time remaining to be 
simulated. If so, it schedules an additional cycle. If there is not, no further events are 
scheduled and the simulation will tenninate. 

Schedule Demand: A demand event is immediately scheduled at a random time 
between 0 and t’ time units in the future using a uniform random distribution. The event 
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then decrements the number of events remaining to be scheduled by one. If there are any 
demands remaining to be scheduled, another schedule demand event is scheduled. For 
cases when multiple demands are to be scheduled, the demands could be scheduled out of 
order (e.g., in a 30 day cycle, the random draw for the first demand could be five days 
into the cycle and the random draw for the second demand could be two days into the 
cycle). To ensure correct execution, the event list is sorted in increasing order by time at 
the conclusion of this event. 

Demand: The demand event corresponds to a single item demand. No variables 
are changed in the demand event. This event exists only as a transition point to the 
backorder or issue event. The demand event is required because the times from the 
schedule demand state could have been randomly generated out of time sequence. If on- 
hand inventory is greater than zero, the “issue” event is scheduled. Otherwise, the 
“backorder” event is scheduled. 

Issue : In the issue event, the on-hand balance is decremented by one and the 
“order for stock” event is scheduled if the current on-hand balance is less than or equal to 
the reorder point. 

Backorder. The backorder event causes the number of items currently 
backordered to increment by one. The event then checks to see if an additional order 
needs to be placed. If the current on-hand balance plus the number on-order minus the 
number backordered is less than or equal to the reorder point the “order for stock” event 
is scheduled. 

Order for stock : The order for stock event increments the number of items on 
order by the reorder quantity and schedules the “receive” event. 

Receive : The receive event decreases the number of items on order by the reorder 
quantity and increases the on-hand balance by the reorder quantity. The event then checks 
to see if there are any items backordered; if so, the “clear backorder” event is scheduled. 

Clear backorder. The clear backorder event decreases the number of items 
backordered and the on-hand inventory by one. If on-hand inventory and the number 
backordered are greater than one, the “clear backorder” event is scheduled. 
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Figure 6. CIS Event Graph 



Run 


{L= s + Q} 


t: length of the lead-time 
t': length of the cycle 
s: reorder point 
Q: reorder quantity 
L: inventory level 

a: number of demands in a particular cyde 
BB: total currently backordered 
00: total currently on-order 
time: current time in the event graph 
max: time for simulation termination 



2. Inputs to CIS 

There are a number of parameters that are input into CIS, which are used to 
detennine the behavior of the NUN. The reorder point defines the inventory level at 
which a reorder will be triggered. The reorder quantity defines that number of units of a 
NUN that will be acquired in a single reorder. The cycle length defines the amount of 
time to which the demand probability distribution function refers. Lastly the DPDF 
specifies the number of demands that could occur during a cycle. There are three possible 
ways that the DPDF can be defined: 

• Probability distribution function - The simulation can accept different 
probability distribution functions as input along with the associated 
parameters. The simulation supports the Poisson, negative binomial, 
normal, binomial, gamma, Pareto, Weibull, unifonn and deterministic 
distributions. 

• Empirical distribution - The empirical distribution is input as any number 
of consecutive months with the number of demands per month. The 
simulation generates a uniform random integer variate from 1 to the 
number of months in the empirical distribution. This random variate is 
used to select which month’s demand is sampled for the cycle. 
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• Probability distribution file - While solving the for the optimal reorder 
points, WIOM generates a file listing probabilities for each demand 
quantity over one cycle.. 

The analysis conducted as part of this research will explore the effect of varying 
different values of reorder quantity, reorder point, and lead-time to explore the sensitivity 
of these parameters. 

3. Outputs from the model 

The simulation determines several important statistics during execution. The total 
number of demand events and the total number of times that a demand could not be filled 
by on-hand inventory are recorded and used to determine the fill-rate according to the 
formula in Chapter I. The simulation also records the means and the standard deviations 
of monthly and cycle demand to provide confidence that the simulation is operating 
properly. The minimum and maximum inventory position observed are recorded to keep 
track of extremes during execution. Lastly, statistics concerning the shortest, longest, 
mean and standard deviation of backorder times are recorded as an indication of the 
effectiveness of the reorder point in addition to fill-rate. 

C. DATA SETS 

Five different data files from January 2016 were provided by NAVSUP for 
exploration using the simulation. These data files contain both 60 months of empirical 
monthly demand data as well as the mean and standard deviation of lead time demand for 
the existence of the NIIN. Although not an exhaustive list of all NIINs stocked by 
NAVSUP, these data sets contain a mixture of high-demand and low-demand items 
supporting a variety of platforms. These data sets provided the reorder point determined 
by SPO as well as the inputs required to calculate reorder points using both WIOM and 
the SC. Table 2 summaries the characteristics of each data set. 


19 



Table 2. Description of Data Sets Provided by NAYSUP 



Total NIINs 

Budget 

Normal 

Binomial 

Poisson 

Neg. Binomial 

Gamma 

Determinstic 

S5S8 

249 

$58,642 

12 

33 

17 

88 

99 

0 

Non-nuclear Consumable 

2231 

$17,189,771 

150 

542 

140 

847 

552 

0 

Non-nuclear Repairable 

7200 

$170,019,778 

1 

2251 

714 

3174 

1060 

0 

Aviation Consumable 

1516 

$7,923,000 

155 

239 

90 

527 

505 

0 

Aviation Repairable 

9601 

$887,486,000 

703 

2030 

74 

2575 

4210 

9 


This table describes the distribution of NIINs according to the rules of the parametric fit 
described in Chapter II. The budget for each category is for the value of safety stock 
allowed for one year. 


To calculate the reorder points using WIOM, the data sets were input into the 
WIOM Excel graphical user interface. The budget solve options were set to solve using 
MIP with a 1% optimality gap or a two hour runtime limit. Separate WIOM runs were 
conducted to find the reorder points using the empirical data files and using the 
parametric estimations. The resulting output files from WIOM were used as input into the 
simulation. 

The reorder points for the SC were detennined by using the aforementioned 
formula. 

reorder point = mean demand over lead-time + Wvariance of the lead-time demand 


To find the solution for t, a Python script was created. The script takes four inputs 
from the user: the name of a NAVSUP provided wholesale data file as input for the 
NIIN’s price, lead-time demand and variance; the name of a WIOM_NIIN file for 
modification; the output filename; and the budget constraint. Upper and lower bounds for 
t are set at 100 and -100, respectively. The script then selects the midpoint between the 
upper and lower bounds and uses that t to find the safety stock. If the safety stock is 
fractional, the safety stock is rounded to the nearest integer. If the safety stock less than 
zero, it is raised to zero. The total yearly cost of safety stock for all NIINs is found by 
multiplying the safety stock level of each NIIN by the price NIIN and then summing for 
all NIINs. If the cost exceeds the budget, the upper bound is lowered to the current t. If 
the cost is less than the budget, the lower bound is set to the current t. New values for t 
are selected until the upper and lower bounds converge to 0.01. 
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A possible source of error exists in the data, which may contribute to erroneous 
results. The error concerns a mismatch between the average demand calculated using the 
empirical distribution and the average demand provided as a parameter in the datasets. 
Since WIOM uses these parameters as inputs into the reorder point calculation, this 
introduces some uncertainty about the optimality of the answer provided by the model. 
As SPO is a black box algorithm, we are unsure if or how this error effects SPO’s 
calculation. For the purpose of this thesis, we assume that the error affects both methods 
equally and a comparison between the two can still be made. 
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IV. ANALYSIS 


This chapter provides the significant results from the simulation study. In the first 
section, the effects of different length warmup times are examined to determine the 
appropriate simulation length for further experimentation. In the second section, fill-rates 
are compared across different data sets using the different reorder point picking methods. 
The third section examines the mean length of time a NUN is backordered resulting from 
the different methods. Section four looks at WIOM’s ability to estimate fill-rates for 
different demand distributions. In the last section, the effect of varying the lead-time is 
examined. This analysis uses the January 2016 input files provided by NAVSUP. The 
results are highly dependent upon the input values and may not be representative of the 
results using different inputs. 

A. EFFECTS OF WARMUP ON THE RESULTS 

Three different trials are used to explore the effects of warmup on the simulation 
results. Following Welch’s (1983) procedure, five different runs are conducted on each 
dataset. Each time an event occurs, the fill-rate is calculated and recorded. Plots of each 
fill-rate are constructed and the convergence point across the trials is observed. This 
convergence point represents the time when the simulation reached steady state and 
wannup effects are no longer affecting the fill-rate. 

The initial five replications were conducted using the S5S8 dataset. The S5S8 
dataset was selected for two reasons. The first reason was that all of the different demand 
probability distributions were represented. The second reason was that it was found that 
the amount of data recording required to plot the fill-rate graphs was such that using one 
of the larger datasets was computationally prohibitive. Demands were simulated for each 
NIIN for one million days and the fill rate over all NIINs was recorded following each 
event in the simulation. 

As shown in Figure 7, the fill-rates for the replications reach steady state around 
day 400,000 in the simulation. Therefore, the simulation was repeated using a 400,000- 
day warmup period. Figure 8 shows that the simulation reaches steady state more quickly 
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with the warmup period; however, the overall fill-rate is not affected. Even with the 
warmup period, the fill-rate is not in steady state immediately at the beginning of the 
simulation. This is an expected behavior. Although starting biases have been removed, 
the fill-rate calculation is a running average of binary events, and a demand is either met 
via an issue from stock or the demand is backordered. It will take some number of events 
to occur for the fill-rate to reach steady state. 


Figure 7. S5S8 Fill-rate for One Million Days, No Warmup 



Five replications of run length one million days are run using the S5S8 data set. The trials 
converge to steady state about day 400,000. 
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Figure 8. S5S8 Fill-rate for 1.4 Million Days, 400,000 Day Warmup 



Time (days) 


Five replications with run length 1,400,000 days are run using the S5S8 data set. A 
warmup period of 400,000 days is specified. In this case, the trials converged more 
quickly than without the warmup period; however, the steady state fill rate observed did 
not change. 

These set of results suggested two things. The first was that one million days was 
sufficiently long in order to observe a reasonable accurate measurement of the fill-rate of 
the system. The second was that a warmup period was not necessary in this simulation 
with this number of days simulated to find a good estimate of the fill-rate. To further 
solidify these claims, two additional cases were tested. 

The next case used the one hundred NIINs from the non-nuclear consumable 
dataset with the highest variance. Those NIINs with a high variance may take longer to 
converge than those NIINs with a smaller variance. Again, two sets of trials each with 
five replications were conducted. The first trial was one million days long without a 
warmup period. As with the previous dataset, Figure 9 shows that fill-rate reached steady 
state in about 400,000 days. In Figure 10, using a 400,000 day warmup period to remove 
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Fill-rate 


starting biases did not have an effect on the final fill-rates. In order to be completely 
confident that warmup periods can be neglected, one more set of experiments were run. 


Figure 9. Non-nuclear Consumables: 100 Most Variable, No Warmup 



Five replications are conducted using the 100 NIINs with the highest variance from the 
non-nuclear consumable data set. The fill rate reaches steady state at 400,000 days 
without a warmup period. 
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Fill-rate 


Figure 10. Non-nuclear Consumables: 100 Most Variable, Warmup 



Time (days) 


Five replications are conducted using the non-nuclear consumable data set with a 400,000 
day warmup period. The fill rates reach steady state after 300,000 days of observations 
and the steady state fill rate value has not changed from the no-warmup period case. 


In a modification of the second set of trials, the last set selected the one hundred 
NIINs with the largest variance to mean ratios. Once again, the non-nuclear consumable 
dataset was selected. Such ratios for this subset ranged from 2.5 to 14.2. 
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Figure 11. Non-nuclear Consumables: 100 Flighest Variance to Mean, No 

Wannup 



Five replications were conducted using 100 NIlNs with the highest variance to mean 
ratios from the non-nuclear consumable data set. As we observed before, steady state was 
achieved after 400,000 days. 
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Figure 12. Non-nuclear Consumables: 100 Highest Variance to Mean, 

Warmup 



Time (days) 


Five replications are conducted using a 400,000-day warmup period with 100 NIINs with 
the highest variance to mean ratios from the non-nuclear consumable data set. The steady 
state value of fill rate has not changed from the no-warmup case. 


Since a warmup period did not seem to affect the fill-rates observed in any of the 
three datasets, a warmup period was not used for future experimentation. 

B. COMPARISON OF FILL-RATES ACROSS DATA SETS 

The most important question this thesis seeks to explore concerns the 
effectiveness of the three different methods (WIOM, SPO and SC) in achieving high fill- 
rates. To explore this question, four different datasets were explored: aviation 
consumables, aviation repairables, maritime consumables and maritime repairables. The 
methods were applied to each dataset in order to find the reorder point for each NIIN. 
Ten replications of the simulation are conducted for each method’s reorder points using 
both the empirical and parametric methods for scheduling demands. In general, a larger 
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number of replications would be preferred. However, even taking advantage of parallel 
processing, each additional 10 replications would take seven days of processing time to 
accomplish. Time limitations did not allow for these additional runs to occur. 

The usual problems that occur from a limited number of runs were offset by 
choosing run lengths that were long enough that the simulation reached steady state. This 
allowed the simulation to overcome the initial transient problem. Additionally, as the 
simulation is in steady state, we would not expect significant changes in the value of the 
output. The observed standard deviations between the 10 runs on each dataset were very 
small and there is no reason to believe additional runs would have any effect on the 
overall results. 


1. Overall Results 


A general formula for calculating fill rate is presented in Chapter 1. Here we show 
three different refinements to the fill rate calculation: overall fill rate, average fill rate, 
and weighted fill rate. Overall fill rate is defined as follows for NIINs i: 


fill rate 


overall 


y demands,. - y backordered,. 

_ i _ i _ 

y demands, 


Additionally, the average fill-rate for NIINs i is calculated as: 


fill rate 


average 


y fill rate,. 

_ i _ 

total number of NIINs 


Finally, weighted fill rate for NIINs i is: 

lead time demand, 


y fill rate,. 


fill ^weighted = 


lead time, 


X 


lead time demand, 
lead time 


Overall fill rate and average fill rate are affected by outliers but are sensitive in different 
ways. In the average fill-rate calculation, low demand items have an outsized effect on 
the value since all NIINs are weighted equally regardless of demand. The overall fill-rate 
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calculation essentially weights the NIINs relative to their demand but allows the effects 
of low demand NIINs to be dominated by the results of the high demand NIINs. An 
example of these effects can be seen in Table 3. The weighted fill rate is a similar to the 
overall fill rate in that it takes the magnitude of the number of items demanded into 
account. The difference between the two calculation is that the weights in weighted fill 
rate are calculated using the mean lead-time demand where the overall fill rate uses the 
observed demand in the simulation. The results of all three calculations are presented to 
provide an overall picture of how well the methods satisfy demand. It is important to note 
that WIOM only estimates weighted fill rate. 


Table 3. Sample Fill-rate Calculation 


NUN, 

Demand, 

Backordered; 

Fill Rate,. 

1 

1 

1 

0.00% 

2 

10 

1 

90.00% 

3 

20 

3 

85.00% 

4 

1000 

0 

1.00% 


overall fill-rate: 99.61% 

average fill-rate: 44.00% 


This table shows the difference between the overall fill-rate and the average-fill rate. The 
overall fill-rate is heavily influenced by the high demand of item 4. In contrast, the 
average fill-rate is heavily influenced by the single failure to fill of NUN 1. 


Tables 4 and 5 summarize the simulated fill rates found using the empirical 
demand distributions and the parametric fit, respectively. In these tables, the mean and 
the standard deviation represent the values found across the 10 replications, not the 
differences between NIINs in a single replication. 
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Table 4. Fill-rate Results Using Empirical Demand 


Overall Fill 

WIOM 
SPO 
SC 

Weighted Fill 

WIOM Projected 
WIOM 
SPO 
SC 

Average Fill 

WIOM 
SPO 
SC 

Four datasets are used to test the fill-rate results using the empirical demand distribution. 
In one of the four cases, the weighted fill-rate observed is nearly identical to the WIOM 
estimate. In ten out of the twelve cases, significantly higher fill-rates were simulated 
using the reorder points selected by WIOM that those selected by SPO or SC. 


Empirical Distribution 


| Aviation Consumables 

Aviation Repairables 

Maritime Consumable 

Maritime Repairable 

Mean 

Standard Dev. 

Mean 

Standard Dev. 

Mean 

Standard Dev. 

Mean 

Standard Dev. 

0.72479 

0.00035 

0.72893 

0.00016 

0.67207 

0.00021 

0.73194 

0.00032 

0.33623 

0.00115 

0.64419 

0.00020 

0.68903 

0.00051 

0.51633 

0.00034 

0.34462 

0.00088 

0.58166 

0.00000 

0.59064 

0.00026 

0.41186 

0.00001 


0.89000 


0.91000 


0.81000 


0.81000 


0.72849 

0.00026 

0.72902 

0.00017 

0.66962 

0.00021 

0.80669 

0.00018 

0.34413 

0.00038 

0.64422 

0.00024 

0.68789 

0.00041 

0.51774 

0.00051 

0.34829 

0.00029 

0.60493 

0.00056 

0.58770 

0.00069 

0.41332 

0.00028 


0.94328 

0.00013 

0.85334 

0.00012 

0.73832 

0.00027 

0.80682 

0.00012 

0.50808 

0.00037 

0.65112 

0.00014 

0.57650 

0.00005 

0.46873 

0.00012 

0.50057 

0.00042 

0.64345 

0.00025 

0.51627 

0.00015 

0.45026 

0.00002 


The first thing to notice about the results under the empirical demand distribution 
is how small the standard deviations are for the outcome of the 10 runs. This adds 
additional weight to the suggestion that the run length of the simulation is appropriate to 
reach steady state. In the maritime repairable case, WIOM was able to estimate the 
average fill-rate achieved through simulation with only a 1% deviation. However, in the 
other three cases, WIOM over-estimated the fill rate up to a 19% difference. 

In all WIOM cases, the average fill-rate is higher than the overall fill-rate. This 
indicates the WIOM is doing a better job selecting reorder points for low demand items 
(weighted more heavily in the average fill-rate calculation) than for high demand items 
(weighted more heavily in the overall fill-rate calculation). In contrast, the fill-rates 
achieved by SPO are higher (in two of four cases) for the overall fill-rate suggesting that 
SPO is favoring higher demand items at the expense of filling low demand orders. It is 
unclear why the fill rates achieved by SPO are so poor for the aviation consumable data. 
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Table 5. Fill-rate Results Using Parametric Fit 


Overall Fill 

WIOM 
SPO 
SC 

Weighted Fill 

WIOM Projected 
WIOM 
SPO 
SC 

Average Fill 

WIOM Projected 
WIOM 
SPO 
SC 

Four data sets are used to find the fill rates resulting from the use of the three different 
methods to select reorder points. The reorder points selected by WIOM resulted in higher 
fill rates than the reorder points select by the other two methods in twelve out of twelve 
cases. 


Parametric Fit 


)n Consur 

Aviation Repairables 

Maritime Consumable 

Maritime Repairable 

Mean 

Mean 

Standard Dev. 

Mean 

Standard Dev. 

Mean 

Standard Dev. 

0.87746 

0.74557 

0.00012 

0.75913 

0.00069 

0.75267 

0.00035 

0.32281 

0.64629 

0.00006 

0.68731 

0.00009 

0.51581 

0.00009 

0.30025 

0.64483 

0.00008 

0.58739 

0.00005 

0.40854 

0.00008 


0.92000 

0.91000 


0.86000 


0.88000 


0.87705 

0.74716 

0.00046 

0.75993 

0.00021 

0.75337 

0.00017 

0.32277 

0.65092 

0.00032 

0.68599 

0.00007 

0.57170 

0.00018 

0.30041 

0.64493 

0.00009 

0.65273 

0.00014 

0.40858 

0.00011 


0.95793 

0.92770 


0.85054 


0.84677 


0.94348 

0.84166 

0.00046 

0.79086 

0.00021 

0.80948 

0.00017 

0.50787 

0.65137 

0.00032 

0.57923 

0.00007 

0.46916 

0.00018 

0.49906 

0.68362 

0.00009 

0.51444 

0.00014 

0.45011 

0.00011 


WIOM is more accurate when using the parametric fit than it is using the 
empirical distributions. The smallest difference between the WIOM estimate and the 
simulated fill rate was found to be 4.3% in the aviation consumable case but no worse 
than 16% in any case. The simulated fill rates achieved by WIOM are higher than those 
achieved by either SPO or SC across all 12 cases. As with the empirical distribution, the 
fill-rate standard deviation observed between replications is small, indicating that steady 
state is achieved for all cases. 

2. Fill-rates Achieved by NUN 

While the mean fill-rates give an overall picture of the effectiveness of the 

different methods, examining the distribution of fill-rates inside of the means gives an 

additional indication of the performance of the reorder point calculations. For example, 

do a few high fill-rate NIINs offset several low fill-rate NIINs or are the fill-rate values 

consistent around the mean? Two aviation data sets are presented to answer this 

question. The fill-rates achieved by the aviation consumable data set are found in Figure 

13, the best performing case. The fill-rates achieved by the aviation repairable data set are 
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found in Figure 14, the worst performing case. The remaining graphs can be found in 
Appendix B. 


Figure 13. Aviation Consumables - Fill-rates Achieved by NUN (Empirical) 
Aviation Consumables - Fill-rates Achieved by NIIN (Empirical) 

1400 


1200 



Fill-rate Percentage 


■ WIOM ■ SPO BSC 

These are the results by NIIN from the aviation consumable data set using the empirical 
distribution. The reorder points selected by WIOM achieved very high fill rates with few 
NIINs in the lower percentages. In contrast, both the SC and SPO had a number of poor 
performing NIINs offset by an equal number of high performing NIINs. 
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Figure 14. Aviation Repairables - Fill-rates Achieved by NUN (Empirical) 
Aviation Repairables - Fill-rates Achieved by NIIN (Empirical) 

6000 



Fill-rate Percentage 
■ WIOM BSPO BSC 


These are the results by NUN from the aviation repairable data set using the empirical 
distribution. 

It is important to note that the target fill rate for the NIINs in the data sets are 
95%. Any NIINs with a fill rate exceeding 95% is not desirable if it comes at the cost of 
another NUN failing to reach 95%. In aviation consumables data set (Figure 13), WIOM 
is able to achieve the fill-rate goal while minimizing the number of NIINs below the 95% 
threshold. Both SPO and SC overbuy safety stock for a few NIINs causing them to 
exceed 95% but then have many more NIINs that fail to reach that threshold. 

As shown in Figure 14, WIOM experiences a similar phenomenon in the aviation 
repairable data set. One possible explanation for the difference in behavior between the 
two data sets may be the price of the NIINs. The average cost of a NIIN from the aviation 
consumable data set is $3,066 while the average cost of a NUN from the aviation 
repairable data set is $80,144. It may not be possible for any of the methods to fit the 
more expensive items into the safety stock budget. 
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3. Selection of Demand Probability Distributions 

When using the parametric distribution, a probability distribution function is 
assigned to a particular NUN based on a parametric fit with simple rules that take the 
NIIN’s demand mean and standard deviation as input. Since there are many different 
distributions that could be assigned that have the same mean and standard deviation, it is 
import to check to see if any of the distributions are causing WIOM to consistently over 
or under-estimate the fill-rate for a NIIN. The graphs of the distribution of fill rates for 
the aviation repairable data set are found in Figures 15 and 16. The graphs for the 
remaining data sets are located in Appendix C. 


Figure 15. Aviation Repairables - WIOM Fill-rate Difference for All 

Distributions 


Aviation Repairables - WIOM Fill-rate Difference 
(Parametric Fit) 
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Predicted Fill-rate - Simulated Fill-rate 


The majority of WIOM’s estimated fill rates are within 2% of the simulated fill rates. 
However, when an estimate and simulated value do not match, it is usually because 
WIOM over-estimated the fill rate. 
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Figure 16. Aviation Repairables - WIOM Fill-rate Difference by Distribution 


Aviation Repairables - WIOM Fill-rate Difference 
by Distribution (Parametric Fit) 
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Predicted Fill-rate - Simulated Fill-rate 


■ Binomial ■ Normal ■Neg.Bin. ■ Poisson ■ Gamma ■ Determ. ■ Empirical 


When the Poisson distribution can be used in a parametric fit, the resulting estimated fill 
rate is very close to the simulated value. The binomial, normal and negative binomial also 
do well. The gamma distribution is less accurate than the other distributions 


The graphs show similar trends across the datasets. They all have a right skew 
demonstrating WIOM’s tendency to over-estimate the fill-rates. The NIINs with demands 
modeled by the Poisson distribution are the most accurate distribution with the smallest 
difference between estimated and simulated fill-rates. The NIINs modeled by the gamma 
distribution show the highest tendency to be least accurately estimated. 

C. AVERAGE TIME BACKORDERED 

While fill rate is the NAVSUP preferred measure of effectiveness, there are some 
limitations to its use as a metric. The most important limitation is that all failures to fill a 
demand are treated equally. That is to say, the item is either available for issue or it is not. 
The fill-rate for an item does not measure how long it will take to fill a backordered item. 
As a result, a demand that could be filled by a reorder arriving tomorrow is penalized 
equally to a demand that cannot be satisfied for one year. This limitation leads to the 
introduction another measure of effectiveness: the average time backordered. If an 
inventory reorder policy leads to a lower average time backordered, the failures to fill an 
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order at the time of the demand will have less of an impact on the customer than an 
alternative inventory policy. 

To explore the backorder space, the mean backorder time is calculated for each 
NIIN across the 10 runs. These means are then compared as shown in Figures 17 and 18 
and Tables 6 and 7. For both the empirical and the parametric demand distributions, the 
reorder points selected by WIOM showed significant improvement in the median 
backorder lengths. This was an unexpected result since WIOM does not make any 
attempt to optimize backorder time. The following table offers an explanation why this 
occurred. In each case, WIOM selected more reorder points at a higher quantity than 
SPO. Regardless of the reason, the higher reorder points caused replenishment orders to 
be placed earlier in the cycle. Since the lead-time for a replenishment order is constant, 
the earlier the replenishment order is placed the sooner in the cycle it will arrive. This 
means any backordered demands will be filled more quickly. Figure 19 provides a 
pictorial example of this phenomena. 
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Figure 17. Backorder Length - Empirical 


Backorder Length - Empirical 



A plot showing the length of backorders in days using reorder points from the three 
methods using the empirical demand distribution. In all cases, the reorder points selected 
by WIOM have a lower median backorder length that those using the other two methods. 


Table 6. Backorder Length-Empirical 



AC-SC 

AC-SPO 

AC-WIOM 

AR-SC 

AR-SPO 

AR-WIOM 

MC-SC 

MC-SPO 

MC-WIOM 

MR-SC 

MR-SPO 

MR-WOIM 

Minimum 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

First Quartile 

93.1 

91.4 

0.0 

254.1 

32.3 

2.8 

260.3 

69.6 

80.6 

83.0 

79.6 

50.8 

Median 

158.7 

155.6 

13.9 

351.5 

65.4 

34.0 

414.0 

151.9 

115.3 

132.4 

132.9 

75.7 

Mean 

173.8 

180.7 

36.3 

377.9 

94.9 

57.1 

453.9 

220.3 

150.8 

145.9 

144.9 

87.3 

Third Quartile 

232.0 

238.5 

60.8 

476.1 

115.5 

77.3 

619.1 

300.2 

179.5 

191.6 

194.2 

111.5 

Maximum 

782.4 

781.3 

323.6 

1863.8 

1711.0 

1038.8 

2907.8 

2616.3 

1072.7 

850.6 

970.2 

694.1 


The backorder length in days using reorder points from the three methods using the 
empirical demand distribution. The reorder points selected by WIOM result in 
significantly shorter backorder lengths than those selected by SPO and SC. 
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Figure 18. Backorder Length - Parametric 


Backorder Length - Parametric 



A plot showing the length of backorders in days using reorder points from the three 
methods using parametric fit. In all cases, the reorder points selected by WIOM have a 
lower median backorder length that those using the other two methods. 


Table 7. Backorder Length-Parametric 



AC-SC 

AC-SPO 

AC-WIOM 

AR-SC 

AR-SPO 

AR-WIOM 

MC-SC 

MC-SPO 

MC-WIOM 

MR-SC 

MR-SPO 

MR-WOIM 

Minimum 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

First Quartile 

88.8 

85.8 

0.0 

28.5 

32.2 

8.0 

91.3 

69.9 

70.8 

82.4 

78.9 

49.8 

Median 

154.5 

153.6 

24.0 

61.9 

65.1 

35.4 

178.4 

150.7 

103.1 

131.9 

132.7 

74.7 

Mean 

170.9 

178.2 

38.1 

88.4 

94.4 

58.5 

247.7 

221.2 

140.9 

145.4 

144.5 

86.4 

Third Quartile 

228.7 

233.0 

66.8 

109.2 

114.7 

78.4 

336.0 

305.7 

166.1 

191.7 

193.6 

110.7 

Maximum 

781.5 

781.6 

244.0 

1711.4 

1711.4 

1052.3 

2479.6 

2614.2 

1054.2 

847.7 

967.0 

712.3 


The backorder length in days using reorder points from the three methods using 
parametric fit. The reorder points selected by WIOM result in significantly shorter 
backorder lengths than those selected by SPO and the SC. 
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Figure 19. Backorder Example 
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Two different inventory policies are represented. The first has a demand of one unit per 
time unit, a reorder lead-time of 4, a reorder quantity of 7 and a reorder point of 2. The 
second has the same characteristics except the reorder point is 1. With the first inventory 
policy, two units of demand experience backorder; the first for two time units, the second 
for one time unit. The resulting average time backordered is 1.5 time units. The second 
inventory policy experiences three backordered units with backorder times of three, two 
and one time units respectively. The resulting average time backordered is two time units. 
Since inventory policy one was able to place its order earlier in the cycle, the mean time 
backordered was lower than that of inventory policy two. 


D. DISTRIBUTION ANALYSIS 

An important part of WIOM’s ability to achieve optimal reorder points is 
accurately estimating the fill-rate for different NIINs. Figure 20 shows the difference 
between the fill rate predicted by WIOM and the fill rate simulated by CIS. If the 
difference between the estimated and simulated fill-rate is positive, the estimate was 
larger than the simulated value. If the difference between the estimated and simulated fill- 
rate is negative, the estimate was smaller than the simulated value. The closer the 

difference is to zero, the more accurately WIOM’s estimated fill-rate matched the 
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simulated fill-rate. As show in Table 8, WIOM was successful in estimating the fill-rates 
with over 25% of the NIINs falling within 2% of the estimate. Of note, the probability of 
over-estimating the fill-rate of a NUN is more than twice the probability of under¬ 
estimating the fill-rate. 


Figure 20. Difference Between Estimated and Simulated Fill-rates 


Difference Between Estimated and Simulated Fill Rates 
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Predicted Fill Rate - Simulated Fill Rate 


■ Maritime Reparable-Par ■ Maritime Consumables-Par ■ Aviation Reparables - Par 

■ Aviation Consumables-Par ■ Maritime Reparables- Emp ■ Maritime Consumables - Emp 

■ Aviation Consumables - Emp ■ Aviation Repairables - Emp 

This figure shows the difference between WIOM’s predicted fill rate and the simulated 
fill rates for all NIINs across all data sets. The smaller the difference between the two 
values the closer to the center of the graph the NUN is plotted. 
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Table 8. Fill-rate Over-estimation and Under-estimation Across All 

Datasets 


NIINs Percentage Definition 


Total 

44610 



[-0.02,0.02] 

18075 

26.40% 

Close estimate 

(0.02,1.00] 

19546 

51.67% 

Ove resti mated 

[-1.00,-0.02) 

6989 

21.92% 

Underestimated 


This table shows the percentage of NIINs across all data sets for which WIOM’s 
estimated fill rate was a close estimate, and overestimate and an underestimate. 


Of particular interest are those NIINs that had estimated values very close to the 
simulated value and those NIINs that had estimated values very far from the simulated 
value. Classification trees can be used to identify the characteristics of the NIINs, which 
could then be used to predict if the NUN was going to have an estimated fill-rate close to 
the simulated fill-rate. The aviation repairables dataset was selected for this analysis due 
to the wide range of differences in estimated and simulated fill-rates. For the remainder 
of this section we will refer to this difference as fill-rate error. 

Two different classification trees are explored. Abbreviations used in the 
classification trees are listed in Table 9. The first classification tree (Figure 21) uses five 
groups to categorize the magnitude of the fill-rate error. The results of this tree are listed 
in Table 11. The second classification tree (Figure 22) divides fill-rate error into two 
groups (Table 12) to separate low error rates from high error rates. The results of this 
classification are listed in Table 13. 

The five factor classification tree is successful in categorizing 75.61% of the 
NIINs. It is interesting to note that as the variance to mean and standard deviation to 
mean ratios increase, the likelihood of misclassification increases. Additionally, the 
distribution used to describe the demand is only a factor in the final split between groups 
1 and 2. 

Similarly, the two factor classification is largely dependent on variance to mean 
and standard deviation to mean ratios. One difference with the five factor classification 
tree is that reorder time is selected as a factor where a shorter lead times were classified 
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as having more accurate fill rate predictions. The successful classification rate using this 
tree is 89.15%. 


Table 9. Abbreviation Used in Classification Trees 


Abbreviation 

Meaning 

DemandMe 

Monthly mean demand 

Distribu 

Probability distribution function for demand 

DSDC.DMC 

Cyclic demand standard deviation / cyclic mean demand 

DSDM.DMM 

Monthly demand standard deviation / monthly mean demand 

DVC.DMC 

Cyclic demand variance / cyclic mean demand 

Mean 

Lead-time mean 

ReorderT 

Number of days fora reorder to arrive 

Var.Mean 

Lead-time variance / lead-time mean 

Variance 

Lead-time variance 


These are the abbreviations used in the classification trees in Figure 21 and Figure 22. 


Table 10. Classification Groupings 



Group 

Range 

Five Factor 

1 

[0,0.02] 

2 

(0.02,0.10] 

3 

(0.10,0.20] 

4 

(0.20,0.30] 

5 

(0.30,1] 

Two Factor 

1 

[0,0.10) 

2 

[0.10,1] 


This table defines the groups used in the two- and five-factor classification trees. 
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Figure 21. Five-Factor Classification Tree for Aviation Repairables 


Aviation Repairables - Empirical 


[yesl 


DSDM.DMM < 3.6 





DVC.DMC < 1.9 


DVC.DMC <1.2 DSDC.DMC < 4.3 



This classification tree uses characteristics of the NUN to predict the difference between 
WIOM’s estimated fill rate and the simulated fill rate. 


Table 11. Five Factor Classification Tree Results 




Predicted 


Group 

1 

2 

3 

4 

5 


1 

5064 

286 

1 

6 

0 

fU 

2 

665 

667 

39 

40 

6 

■M 

U 

3 

418 

120 

203 

77 

19 

< 

4 

149 

10 

27 

223 

73 


5 

292 

0 

0 

42 

879 


If the actual fill-rate error group that a NUN is categorized in is the same as the predicted 
fill-rate error group (e.g., on the diagonal) a correct classification has been made. 
Otherwise, a misclassification has occurred. The classification tree has a misclassification 
rate of 24.39%. 
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Figure 22. Two Factor Classification Tree for Aviation Repairables 

Aviation Repairables - Empirical 



< 





This classification tree uses characteristics of the NUN to predict the difference between 
WIOM’s estimated fill rate and the simulated fill rate. 


Table 12. Two Factor Classification Tree Results 




Predicted 


Group 

1 

2 

(TJ 

1 

6488 

286 

■M 

U 

< 

2 

724 

1808 


If the actual fill-rate error group that a NUN is categorized in is the same as the predicted 
fill-rate error group (e.g., on the diagonal) a correct classification has been made. 
Otherwise, a misclassification has occurred. The classification tree has a misclassification 
rate of 10.85%. 
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E. SENSITIVITY TO -TIME 

The mean amount of time to receive a reorder was the only lead-time information 
provided by NAVSUP. Since the lead-time is the detennining factor as to when a reorder 
arrives, having some variability in the lead-time may affect the observed fill-rates. To test 
the potential sensitivity to the factor, the empirical aviation consumable dataset was 
selected. The lead-time was assumed to be normally distributed with a mean as provided 
by NAVSUP and different levels of variance. Table 11 shows the results of this analysis. 
While there was a slight decrease in the simulated fill-rate as variability increased, the 
overall effect was negligible. However, assuming that the normal distribution applies in 
this case may not be correct. Additional information about the true nature of the lead-time 
distribution may lead to different results. 


Table 13. Sensitivity of the Fill-rate to Stochastic Lead-time 


Distribution 

Std Dev as % of Mean 

Simulated Fill Rate 

Normal 

0 

0.9432 

Normal 

10 

0.9431 

Normal 

20 

0.9430 

Normal 

30 

0.9430 


This chart shows the sensitivity of the fill rate to various stochastic lead-times. Under the 
normal distribution, there is little change in the simulated fill rate. Other distributions 
may yield different results but no data was available to suggest other distributions or their 
parameters. 
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V. CONCLUSIONS AND RECOMMENDATIONS FOR FOLLOW- 

ON STUDIES 


This thesis provides qualitative and quantitative infonnation to help NAVSUP 
explore the effects of using WIOM, SPO and the SC to set reorder points across a variety 
of NIINs. This section summarizes the findings and provides some recommendations for 
future work. 

A. CONCLUSIONS 

The overarching goal of CIS is to produce a model that produces an estimated fill- 
rate based on a demand probability distribution and various parameters that describe the 
NIIN. CIS is able to draw a number of different conclusions about the three different 
methods for determine a reorder point: 

• When considered as an entire system, WIOM’s fill-rate estimations range 
between 1 and 18% of the simulated values. WIOM’s fill-rate estimations 
are closer to the simulated value when the parametric fit is used. 

• The reorder points selected by WIOM resulted in higher average fill-rates 
than those selected by SPO and the SC. 

• SPO’s tendency to select higher reorder points for high demand items had 
two consequences. The first is a higher overall fill-rate when compared 
with WIOM in some cases. The second is a higher mean backorder length 
when compared with WIOM. 

• The SC consistently underperformed both WIOM and SPO. 

• A demand probability distribution fit to a gamma probability distribution 
function is more likely to have a large deviance between the WIOM 
estimated fill-rate and the simulated fill-rate than any other probability 
distribution. The gamma function should be avoided whenever possible. 

One of the advantages of CIS is that it offers a level playing field to evaluate the 
different methods for finding the reorder point. The only difference between simulations 
of the different methods was the reorder points themselves; how those reorder points 
were determined was unimportant and ignored by CIS. This provides an unbiased 
comparison of the three methods and can help to inform a decision regarding future 
reorder point setting algorithms. 
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Another advantage of CIS is the ability to calculate backorder statistics for 
comparison between the three methods. This ability was not previously available in 
previous analysis and offered additional insight into the differences in the outcomes 
between SPO and WIOM. 

B. RECOMMENDATIONS FOR FOLLOW-ON RESEARCH 

While this thesis sought to explore many different aspects of the reorder point 
establishment problem, there are many opportunities for follow-on research. This thesis 
briefly explored the sensitivity of the WIOM selected reorder points to variability in lead- 
time, a more representative distribution could be found which may lead to different 
results. Additionally, sensitivity to other input parameters could be explored. 

One of the assumptions of the simulation is that demands for items are always for 
a single quantity. While this is true for the repairable datasets, it is usually a false 
assumption for the consumable datasets. Relaxing this assumption and using probability 
distributions for the number of items in a single demand event would allow for a more 
realistic simulation. 

This thesis considered NIINs collectively regardless of their individual properties. 
NIINs with very low demand and high variance were grouped with NIINs with very high 
demand and low variance. WIOM has an option to use level setting strategy indicators 
(LSSI) where NIINs of similar properties are grouped together. Future work could 
consider the characteristics of a NIIN that should put it in one LSSI or another. 
Additionally, a comparison could be made between the results of fill rates achieved using 
LSSIs with the fill rates of the single category of NIINs. 

Lastly, both WIOM and the simulation are heavily dependent upon the probability 
distributions for the demand. The parametric fit for the demand distribution function 
relied solely on the lead-time mean and standard deviation. Future work could examine 
these fits to detennine if they could be refined with additional infonnation about the 
NIIN. 
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APPENDIX A. SUPPLY TERMS 


Backorder —When a demand for a NUN cannot immediately be satisfied due to 
insufficient stock on hand, the request for the item is placed into a queue for fulfillment 
upon receipt of a replenishment order. A demand in this state is said to be backordered. 


Cycle —A measure of the amount of time defined by 


t : = 


f 

max 1, 

V 


* A 

T 

a-J 


where t i is 


the lead-time, x i is lead-time demand and Q i is the reorder size. 


Demands —A request from an end user for an item. In this thesis, all demands are 
assumed to be for a single unit of the item. 

Fill-rate —The percentage of demands that can be immediately be satisfied by an 
issue from on-hand inventory. 

Inventory position —The sum of the number of items in inventory plus the 
number of items on order minus the number of items backordered. 

Lead-time —The amount of time between an order being placed for additional 
stock and the receipt of those items. 

NUN—National identification number. A 9-digit subset of the NSN. 

Reorder point (s) —The answer to the question of when to order additional units 
for inventory. When the inventory position reaches the reorder point, an order is placed 
for additional reorder quantity of the item, which will be delivered at the end of the lead- 
time. 


Reorder quantity (Q) — The amount of an item ordered during a reorder. 

Safety stock —Extra inventory kept on hand to act as a hedge in case actual 
demand exceeds expected demand during the lead-time. A backorder occurs when a 
demand is processed and safety stock is exhausted. 


51 



THIS PAGE INTENTIONALLY LEFT BLANK 


52 



APPENDIX B. FILL-RATE GRAPHS BY DATASET 


Figure 23. Maritime Consumables: Fill-rates Achieved by NIIN (Empirical) 
Maritime Consumables - Fill-rates Achieved by NIIN (Empirical) 
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Figure 24. Maritime Repairables: Fill-rates Achieved by NUN (Empirical) 
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Figure 25. Aviation Consumables: Fill-rates Achieved by NUN (Parametric) 
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Figure 26. Aviation Repairables: Fill-rates Achieved by NUN (Parametric) 
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Figure 27. Maritime Consumables: Fill-rates Achieved by NIIN (Parametric) 
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Figure 28. Maritime Repairables: Fill-rates Achieved by NUN (Parametric) 
Maritime Repairables - Fill-rates Achieved by NIIN (Parameteric) 
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APPENDIX C. FILL-RATE DISTRIBUTION GRAPHS 


Figure 29. Aviation Repairables: WIOM Fill-rate Difference for All 

Distributions 
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Figure 30. Aviation Repairables: WIOM Fill-rate Difference by Distribution 
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Number of NIINs s= Number of NIINs 


Figure 31. Maritime Consumables: WIOM Fill-rate for All Distributions 
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32. Maritime Consumables: WIOM Fill-rate by Distribution 
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Figure 33. Maritime Repairables: WIOM Fill-rate Difference for All 

Distributions 
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Figure 34. Maritime Repairables: WIOM Fill-rate Difference by Distribution 
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